Most video applications seek the highest possible perceptual quality given the bit rate constraints. For instance, in a low bit rate application such as a videophone system, a video encoder may provide higher quality by eliminating the strong visual artifacts at the regions of interest that are visually more important. On the other hand, in a high bit rate application, visually lossless quality is expected everywhere in the pictures and a video encoder needs to also achieve transparent quality. One challenge in obtaining transparent visual quality in high bit rate applications is to preserve details, especially at smooth regions where loss of details are more visible than at non-smooth regions because of the texture masking property of the human visual system.
Increasing the bit rate is one of the most straightforward approaches to improve the quality. When the bit rate is given, an encoder manipulates its bit allocation module to spend the available bits where the most visual quality improvement can be obtained. In non-real-time applications such as DVD authoring, the video encoder can facilitate a variable-bit-rate (VBR) design to produce a video with a constant quality over time on both difficult and easy contents. In such applications, the available bits are appropriately distributed over the different video segments to obtain constant quality. In contrast, a constant-bit-rate (CBR) system assigns the same number of bits to an interval of one or more pictures despite their encoding difficulty and produces visual quality that varies with the video content. For both VBR and CBR encoding systems, an encoder can allocate bits according to perceptual models within a picture. One characteristic of human perception is texture masking, which explains why human eyes are more sensitive to loss of quality in smooth regions than in textured regions. This property can be utilized to increase the number of bits allocated to smooth regions to obtain a high visual quality.
The quantization process in a video encoder controls the number of encoded bits and the quality. It is common to adjust the quality through adjusting the quantization parameters. For illustrative purposes, we refer to the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) standard/International Telecommunication Union, Telecommunication Sector (ITU-T) H.264 Recommendation (hereinafter the “MPEG-4 AVC Standard”) as a baseline example to explain the quantization process and improvements that may be had by implementing the present principles over the MPEG-4 Standard. However, it is to be appreciated that the present principles are not limited solely to the MPEG-4 AVC Standard and, thus, may be applied to other video coding standards, recommendations, and extensions thereof, while maintaining the spirit of the present principles.
Mathematically, in the encoder the transformed coefficients are quantized as follows:
                              Z          =                                    ⌊                                                                                        W                                                        q                                +                s                            ⌋                        ·                          sgn              ⁡                              (                W                )                                                    ,                            (        1        )            where W is the transformed coefficient and is quantized to a quantization level Z. Here, q is the quantization step size and s is the quantization rounding offset. The function └.┘ rounds a value to the nearest integer and sgn (.) returns the sign of a signal. When the quantization matrix is applied, the coefficients are scaled first before the quantization process at the encoder. The range of W where it is quantized to 0 is called the deadzone. In this particular case, the deadzone is Δ=(1−s)×q×2. At the decoder, the quantization level Z is reconstructed to the signal W′. This is called inverse quantization and is described mathematically as follows:W′=q·Z.  (2)
The syntax in the MPEG-4 AVC Standard allows q to be different for each macroblock (MB). The value of the quantization step size q is selected from the ones indexed by parameter QP, an integer in the range of 0-51. The rounding offset parameter s, is not involved in the inverse quantization and the encoder has the flexibility of setting it to any value.
Existing MPEG-4 AVC video encoders assume the quantization rounding offset is constant and only adjust the quantization step size to adjust the number of bits and therefore the quality.